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The  5  W’s  of  MS-B  TRA  SvjTEMS  engineering 

(what,  who,  why,  when,  where)  :  INTEGRRTION 


What:  A  TRA  is  a  systematic,  metrics-based  process  that  assesses  the 
maturity  of,  and  the  risk  associated  with,  critical  technologies  to  be  used  in 
Major  Defense  Acquisition  Programs. 

Who:  It  is  conducted  by  the  Program  Manager  (PM)  with  the  assistance  of 
an  independent  team  of  subject  matter  experts  (SMEs). 

Why:  Required  by  DOD!  The  TRA  is  used  by  the  Assistant  Secretary  of 
Defense  for  Research  and  Engineering  (ASD(R&E))  as  part  of  the  basis  to 
advise  the  Milestone  Decision  Authority  (MDA)  whether  the  technologies  of 
the  program  have  acceptable  levels  of  risk. 

•  When:  MS-B  Review  is  the  decision  gate  between  the  Technology 
Development  (TD)  Phase  and  the  Engineering  &  Manufacturing 
Development  (EMD)  Phase.  One  of  the  purposes  of  the  TD  Phase  is  to 
reduce  technology  risk  and  demonstrate  critical  technologies  on  prototypes 
in  relevant  environments. 

Where:  During  the  MS-B  Review,  the  program  is  required  to  show  that  the 
system/subsystem  prototypes  using  the  technologies  have  been 
demonstrated  satisfactorily  in  a  relevant  environment  (TRL6). 
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•  Latest  TRA  Guidance  requires  comprehensive  technical  risk  assessment. 

•  The  assessment  should  be  based  on  objective  evidence  gathered  during 
events,  such  as  tests,  demonstrations,  pilots,  or  physics-based  simulations. 
Based  on  the  requirements,  identified  capabilities,  system  architecture, 
software  architecture,  concept  of  operations  (CONOPS),  and/or  the  concept 
of  employment,  the  SME  team  will  evaluate  whether  performance  in  relevant 
environments  and  technology  maturity  have  been  demonstrated  by  the 
objective  evidence. 

•  If  demonstration  in  a  relevant  environment  has  not  been  achieved,  the  SMEs 
will  review  the  risk-mitigation  steps  intended  by  the  PM  and  make  a 
determination  as  to  their  sufficiency  to  reduce  risk  to  an  acceptable  level. 

•  TRLs  will  be  used  as  a  knowledge-based  standard  or  benchmark  but  should 
not  substitute  for  professional  judgment  tailored  to  the  specific  circumstances 
of  the  program. 
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TRA  PROCESS  TIMING 
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TRATime  Estimate 
versus  Actual 


Technology  Maturity 

Estimate 

Actual 

Assessment  Tasks 

(days) 

(daysj 

Project  planning 

15 

15 

Scope  of  work 

3 

3 

Collect  project  data 

20 

50 

Extract  technology 

15 

25 

Form  SMETeam 

5 

10 

Link  technology  &  KPP 

2 

3 

Establish  CTE criteria 

5 

3 

Initial  CTE  screening 

5 

50 

PM/ASAIALT)  Review 

10 

OSD  CTE  deliberation 

20  parallel 

CTE  &  TMA  process  brief 

10 

Co  1 1  e  ct  TR  L6  req  u  irem  ent  s 

20 

SMETailorTRL  checklist 

8 

10 

Combine  CTE TRL  checklist  & 
TRL6  technical  requirements 

4 

5 

SMETRL6  metrics  review 

5 

PM  TRL6  metrics  review 

10 

Test  data  collection 

95  parallel 

Organize  CTE  data 

15 

.Assess  CTE  technical  risks 

15 

20 

Co  1 1  e  ct  SM  E  risk  asse  ssm  en  ts 

4 

PM  review  of  assessments 

5 

Finalize  CTE  assessments 

10 

Fill  in  TRL  checklist 

95  parallel 

Calculate  TRL/MR1/PRL 

20 

5 

Summarize  all  assessments 

10 

10 

ASA|ALT)TRL  &  BOE  review 

10 

Write  draft  TMA  report 

50 

30 

Internal  reviews 

5 

5 

Finalize  TMA  Report 

10 

10 

Total 

192d=38wk 

353d=71wk 

=9m 

=16m 
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----- 


The  original  estimate  was  9  months  to 
complete  the  whole  TRA  process. 

The  actual  time  was  16  months. 


Lesson:  Many  collaboration  reviews  and  information 
organization  tasks  were  not  included  in  the  original 
planning.  If  included,  the  project  took  1.8  times  longer.  A 
better  estimate  on  TRA  project  timing  should  include  time 
required  to  conduct  reviews  and  information  gathering 
and  organization. 


Lesson:  The  time  required  to  identify  the  technologies  for 
each  contractor  can  be  significantly  reduced  if  the 
information  is  provided  by  the  program  engineers  or  the 
contractors. 
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Step  2 


CDR- Critical  Design  Review 

CDD  -  Capability  Development  Document 

PD  -  Purchased  Description 

TRL  -  Technology  Readiness  Level 


Lesson:  Before  requesting  support  from 
subject  matter  experts  (SMEs)  and 
information  from  the  PM,  develop  a 
flowchart  to  show  the  TRA  process  and  a 
swimlane  chart  to  show  the  detail  TRA  steps 
combined  with  roles  of  each  participating 
organizations.  These  charts  demonstrate 
how  and  when  team  members  have  to  rely 
on  each  other  to  complete  the  TRA  project.. 
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Note:  Additional  requirements  can  be  added  as  requi 
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The  MS-B  Review  mainly  focused  on  the  CTEs  which  are  defined 
by  2  main  criteria. 

1 .  The  system  being  acquired  depends  on  this  technology 
element  to  meet  Key  Performance  Parameters  (KPP)  or  Key 
System  Attributes  (KSA).  (see  the  following  requirement  tree 
and  vehicle  work  breakdown  structure  (WBS)). 

2.  The  technology  or  its  application  is  new  or  in  an  area  that 
poses  major  technological  risk  during  design/demonstration. 

These  criteria  are  expanded  upon  in  the  following  table  to  assist 
the  CTE  identification  process. 
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Critical  Technology  Element  Screening  Criteria 

Y/ 

N 

Y/ 

N 

1)  Critical 

Directly  impact  an  operational  Requirement  threshold 
(KPPor  WBS)? 

Significant  impact  on  an  improved  delivery  schedule? 

Significant  impact  on  program  affordability? 

AND 

2)  New  or  novel 
application 

Is  the  technology  new  or  novel? 

Is  the  technology  modified? 

Tech  repackaged  such  that  a  new  relevant  Environment  is 

realized? 

Expected  to  operate  beyond  design  intention  & 
demonstrated  capability? 

Or 

3)  High 
Risk 

Is  the  technology  in  an  area  that  poses  major 
technological  risk  during  detailed  design  or 

demonstration? 

Unclassified 
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Detail  Design  &  Demonstration  INTEGRATION 


Lesson:  In  order  to  determine  if  there  were  any  major  technological  risks 
associated  with  a  technology  during  detail  design  and  demonstration, 
many  engineering  supporting  documents  have  to  be  reviewed: 

1 .  Engineering  analysis 

2.  Test  results 

3.  Failure  analysis  and  corrective  action  reports  (FACAR) 

4.  Documented  design  risk  and  issue  matrixes 

5.  Requirement  compliance  matrix 

6.  Failure  mode  and  effect  analysis  (FMEA) 

Modeling  and  simulation  analysis 

8.  Boundary  diagrams  and  interface  analysis 

9.  Trade  studies 

10.  Risk  and  issue  discussion  with  PM- J LTV  engineers,  etc. 
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Lesson:  Before  performing  the  assessment ,  a  detailed  metric  has  to  be 
developed  and  agreed  upon  for  the  technologies  between  the  program 
and  IRT.  The  metric  should  contain  specific  detail  technical  reguirements 
for  each  CTE,  verification  methods  and  acceptance  criteria. 
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There  were  a  lot  of  engineering  supporting  documents  for  the  entire  program. 
Only  the  ones  that  were  relevant  to  the  CTEs  and  technology  concerns  needed 
to  be  reviewed.  Having  the  specific  technology  information  organized  allowed 
the  SMEs  to  complete  their  assessments  quickly. 


Lesson:  Rather  than  having  the  SMEs  to  locate  the  engineering  information  they 
need  to  make  the  assessment ,  locate  and  catalogue  all  relevant  technical  data 
(M&S  results ,  test  data  and  reports ,  failure  incidences  and  corrective  actions , 
engineering  analysis ,  etc.)  so  they  can  easily  refer  to  them  while  performing  the 
assessment. 
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1 .  Detail  SME  Assessment  Method  (Test  Based) 
TRL  Calculator  Method  (Tasks  Based) 
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1.  Technical  requirement  identification 

Identify  all  related  CDD  and  PD  engineering  requirements,  acceptance 
criteria  for  the  CTEs  and  IRT  concerns  based  on  TRL6  definition. 


2.  Data  collection 


Collect  information  such  as  technical  requirements,  acceptance  criteria,  test 
data  including  failure  incidence  reports,  failure  analysis  and  corrective  action 
reports,  relevant  CDR  presentations,  risk  and  requirement  compliance  matrices. 

3.  Substantiation  package  preparation 

Catalogue  data  by  technology  and  contractor.  Develop  requirement 
and  test  data  matrix  for  each  technology  so  SMEs  could  find  them 
quickly 

4.  Technical  risk  assessment  and  report 


Based  on  objective  test  data,  engineering  design  and  integration 
supporting  documents,  manufacturing  information  and  SMEs’  relevant 
experience,  write  up  assessments  with  rationale. 
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Lessons  Learned  on  the  SvjTEMS  engineering 

Detailed  SME  Methodology  0 


Lesson:  Failure  to  meet  requirement  does  not  necessarily  mean  TRL  <  6.  In  case 
of  non-compliance,  the  SMEs  will  review  the  corrective  action  taken  and 
determine  if  there  is  an  unacceptable  risk  for  the  EMD  Phase.  A  justification  will 
be  required. 


Lesson:  To  provide  a  more  comprehensive  technical  risk  assessment,  in  addition 
to  Technology  Readiness  Level  (TRL),  it  is  recommended  to  include  Manufacturing 
Readiness  Level  (MRL)  and  System  Readiness  Level  (SRL)  in  the  assessment 
especially  if  there  are  significant  manufacturing  and  integration  risks  or  issues 
during  the  Technology  Development  (TD)  Phase. 


Lesson:  Many  reliability  &  performance  tests,  failure  root  causes  and  corrective 
action  reports  were  not  complete  when  maturity  assessments  were  made.  It  is 
better  to  update  the  assessments  at  the  end  of  the  test  phase  after  test  & 
evaluation  engineers  summarize  the  test  results. 
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•  TRL  Calculator  version  2.2  was  used  for  the  TRL  analysis. 

•  This  version  of  the  Calculator  was  released  by  AFRL  in  May,  2004. 

•  The  original  version  was  released  by  the  Air  Force  Research  Lab 
around  2002. 

•  It  provides  a  snap-shot  of  what  a  technology’s  maturity  level  was  at  a 
given  time. 

•  There  are  three  main  categories  of  tasks  in  maturing  a  technology. 

Technical  (TRL)  -  measures  the  technical  maturity  of  the  technology; 

Manufacturing  (MRL)  -  measures  the  readiness  of  the  production  system 
to  manufacture  the  technology  being  developed;  and 

Programmatic  (PRL)  -  measures  program  management  concerns  of  each 
technology. 

•  For  each  technology,  the  generic  list  of  tasks  has  to  be  tailored  for  the 
assessment. 


The  percent  completion  for  each  task  is  required  with  justification. 
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Lesson:  The  aggregate  TRL  produced  by  the  TRL  Calculator  depends  on  the 
worst  of  the  TRL .,  MRL  and  PRL  rating.  A  low  TRL  may  be  caused  by 
programmatic  elements  rather  than  the  technology  itself.  Detail  interpretation 
of  each  calculated  rating  is  repaired. 
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Jerome  Tzau 

TARDEC  System  Engineering  Group 
6501  E.  11  Mile  R<±,  Bldg  200A,  Mail  Stop  267 
Warren,  Ml  48397-5000 
Office  phone:  (586)  282-0703 
email:  ierome.tzau.civ@mail.mil 
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Technology  Risk  =  the  risk  that  a  technology,  necessary  for  a  capability, 
will  not  mature  within  the  required  time  frame’.  TRL  can  be  used  as  a 
filter  for  assessing  technology  readiness  (i.e.,  maturity  and  feasibility) 
of  new  technologies  and  give  an  indication  of  the  technical  challenge 
ahead. 

•  Technical  Risk  (System):  the  risk  that  a  system  will  not  reach  its 
performance  goals,  development  will  not  be  within  the  specified  time 
frame  and/or  it  will  cost  more  than  estimated  owing  to  difficulties 
experience  with  technical  aspects;  the  risk  associated  with  systems, 
their  integration  with  other  systems  and  their  implementation. 

•  Technical  Assessment:  Technology  assessment  is  a  subset  of 
technical  assessment. 


Risk  Management:  Technology  development  can  be  considered  a 
subset  of  risk  management  and  as  such  should  be  a  primary 
component  of  the  risk  assessment. 


Source:  Technology  Readiness  &  Technical  Risk  Assessment  for  the  Australian  Defense  Origanisation  , 
Terry  Moon,  Jim  Smith,  Stephen  Cook,  2004. 
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Technology  Readiness  Level  Definitions,  Descriptions,  and  Supporting  Information 

TRL 

Definition 

Description 

Supporting  Information 

1 

Basic  principles  observed 
and  reported. 

Lowest  level  of  technology  readiness.  Scientific 
research  begins  to  be  translated  into  applied 
research  and  development  (R&D).  Examples 
might  include  paper  studies  of  a  technology's 
basic  properties. 

Published  research  that  identifies  the  principles  that 
underlie  this  technology.  References  to  who,  where,  when. 

2 

Technology  concept  and/or 
application  formulated. 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be 
invented.  Applications  are  speculative,  and 
there  may  be  no  proof  or  detailed  analysis  to 
support  the  assumptions.  Examples  are  limited 
to  analytic  studies. 

Publications  or  other  references  that  outline  the  application 
being  considered  and  that  provide  analysis  to  support  the 
concept. 

3 

Analytical  and  experimental 
critical  function  and/or 
characteristic  proof  of 
concept. 

Active  R&D  is  initiated.  This  includes  analytical 
studies  and  laboratory  studies  to  physically 
validate  the  analytical  predictions  of  separate 
elements  of  the  technology.  Examples  include 
components  that  are  not  yet  integrated  or 
representative. 

Results  of  laboratory  tests  performed  to  measure 
parameters  of  interest  and  comparison  to  analytical 
predictions  for  critical  subsystems.  References  to  who 
where,  and  when  these  tests  and  comparisons  were 
performed.  Examples  include  "high  fidelity"  laboratory 
integration  of  components. 

4 

Component  and/or 
breadboard  validation  in  a 

laboratory  environment. 

Basic  technological  components  are  integrated 
to  establish  that  they  will  work  together  This  is 
relatively  "low  fidelity"  compared  with  the 
eventual  system.  Examples  include  integration 
of  "ad  hoc"  hardware  in  the  laboratory. 

System  concepts  that  have  been  considered  and  results 
from  testing  laboratory-scale  breadboards(s).  References  to 

who  did  this  work  and  when.  Provide  and  estimate  of  how 

breadboard  hardware  and  test  results  differ  from  the 

expected  system  goals. 
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Technology  Readiness  Level  Definitions,  Descriptions,  and  Supporting  Information 

TRL 

Definition 

Description 

Supporting  Information 

5 

Componentand/or 
breadboard  validation  in  a 

relevant  environment. 

Fidelity  of  breadboard  technology  increases 
significantly.  The  basic  technological  components 
are  integrated  with  reasonably  realistic 
supporting  elements  so  they  can  be  tested  in  a 
simulated  environment.  Examples  include  "high 
fidelity"  laboratory  integration  of  components. 

Results  from  testing  a  laboratory  breadboard  system  are 
integrated  with  other  supporting  elements  in  a  simulated 
operational  environment.  How  does  the  "relevant 
environment"  differ  from  the  expected  operational 
environment?  How  do  the  test  results  compare  with 
expectations?  What  problems,  if  any,  were 
encountered?  Was  the  breadboard  system  refined  to 
more  nearly  match  the  expected  system  goals. 

6 

System/subsystem  model 
or  prototype  demonstration 

in  a  relevant  environment. 

Representative  model  or  prototype  system, 
which  is  well  beyond  that  of  TRL  5,  is  tested  in  a 
relevant  environment.  Represents  a  major  step 
up  1  a  technology's  demonstrated  readiness. 
Examples  include  testing  a  prototype  in  a  high 
fidelity  laboratory  environment  or  in  a  simulated 
operational  environment. 

Results  from  laboratory  testing  of  a  prototype  system 
that  is  near  the  desired  configuration  in  terms  of 
performance,  weight,  and  volume.  How  did  the  test 
environment  differ  from  the  operational  environment? 
Who  performed  the  tests?  How  did  the  test  compare 
with  expectations?  What  problems,  if  any,  were 
encountered?  What  are/were  the  plans,  options,  or 
actions  to  resolve  problems  before  moving  to  the  next 

level? 

7 

System  prototype 

demonstration  in  an 

operational  environment. 

Prototype  near  or  at  planned  operational  system. 
Represents  major  step  up  from  TRL  6  by  requiring 
demonstration  of  an  actual  system  prototype  in 
an  operational  environment  (e.g.,  in  an  aircraft,  in 
a  vehicle,  or  in  space) 

Results  from  testing  a  prototype  system  in  an  operational 
environment.  Who  performed  the  tests?  How  did  the 
test  compare  with  expectations?  What  problems,  if  any, 
were  encountered?  What  are/were  the  plans  options,  or 
actions  to  resolve  problems  before  moving  to  the  next 

level? 
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Technology  Readiness  Level  Definitions,  Descriptions,  and  Supporting  Information 

TRL 

Definition 

Description 

Supporting  Information 

8 

Actual  system  completed 
and  qualified  through  test 

and  demonstration. 

Technology  has  been  proven  to  work  in  its  final 
form  an  under  expected  conditions.  In  almost  all 
cases,  this  TRL  represents  the  end  of  true  system 
development.  Examples  include  developmental  test 
and  evaluation  (DT&E)  of  the  system  in  its  intended 
weapon  system  to  determine  if  it  meets  design 
specifications. 

Results  of  testing  the  system  in  its  final 
configuration  under  the  expected  range  of 

environmental  conditions  in  which  it  will  be 

expected  to  operate.  Assessment  of  whether  it  will 
meet  its  operational  requirements.  What  problems, 
if  any,  were  encountered?  What  are/were  the 
plans,  options,  or  actions  to  resolve  problems 
before  finalizing  the  design? 

9 

Actual  system  proven 
through  successful  mission 
operations. 

Actual  application  of  the  technology  in  its  final  form 
and  under  mission  conditions,  such  as  those 
encountered  in  operational  test  and  evaluation 
(OT&E).  Examples  include  using  the  system  under 
operational  mission  conditions. 

OT&E  reports. 
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Manufacturing  Readiness  Level  Definitions,  Descriptions,  and  Phase 

MRL 

Definition 

Description 

Phase 

1 

Basic  principles  observed 
and  reported. 

Lowest  level  of  technology  readiness.  Scientific  research  begins  to  be  translated  into  applied 
research  and  development  (R&D).  Examples  might  include  paper  studies  of  a  technology's  basic 
properties. 

Pre  Materiel 

Solution 

Analysis 

2 

Manufacturing  concepts 

identified 

This  level  is  characterized  by  describing  the  application  of  new  manufacturing  concepts. 

Applied  research  translates  basic  research  into  solutions  for  broadly  defined  military  needs. 
Typically  this  level  of  readiness  includes  identification,  paper  studies  and  analysis  of  material 
and  process  approaches.  An  understanding  of  manufacturing  feasibility  and  risk  is  emerging. 

Pre  Materiel 

Solution 

Analysis 

3 

Manufacturing  proof  of 
concept  developed 

This  level  begins  the  validation  of  the  manufacturing  concepts  through  analytical  or  laboratory 
experiments.  This  level  of  readiness  is  typical  of  technologies  in  Applied  Research  and 

Advanced  Development.  Materials  and/or  processes  have  been  characterized  for 
manufacturability  and  availability  but  further  evaluation  and  demonstration  is  required. 
Experimental  hardware  models  have  been  developed  in  a  laboratory  environment  that  may 
possess  limited  functionality. 

Pre  Materiel 

Solution 

Analysis 

4 

Capability  to  produce  the 
technology  in  a  laboratory 

environment. 

This  level  of  readiness  acts  as  an  exit  criterion  for  the  Materiel  Solution  Analysis  (MSA)  Phase 
approaching  a  Milestone  A  decision.  Technologies  should  have  matured  to  at  least  TRL  4.  This 
level  indicates  that  the  technologies  are  ready  for  the  Technology  Development  Phase  of 
acquisition.  At  this  point,  required  investments,  such  as  manufacturing  technology 
development,  have  been  identified.  Processes  to  ensure  manufacturability,  producibility,  and 
quality  are  in  place  and  are  sufficient  to  produce  technology  demonstrators.  Manufacturing 
risks  have  been  identified  for  building  prototypes  and  mitigation  plans  are  in  place.  Target  cost 
objectives  have  been  established  and  manufacturing  cost  drivers  have  been  identified. 
Producibility  assessments  of  design  concepts  have  been  completed.  Key  design  performance 
parameters  have  been  identified  as  well  as  any  special  tooling,  facilities,  material  handling  and 
skills  required. 

Materiel 

Solution 

Analysis 
leading  to  a 

MS-A  decision. 

NDII 
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Manufacturing  Readiness  Level  Definitions,  Descriptions,  and  Phase 

MRL 

Definition 

Description 

Phase 

5 

Capability  to 
produce 
prototype 
components  in 
a  production 

relevant 

environment 

This  level  of  maturity  is  typical  of  the  mid-point  in  the  Technology  Development  Phase  of  acquisition,  or  in  the 
case  of  key  technologies,  near  the  mid-point  of  an  Advanced  Technology  Demonstration  (ATD)  project. 
Technologies  should  have  matured  to  at  least  TRL  5.  The  industrial  base  has  been  assessed  to  identify  potential 
manufacturing  sources.  A  manufacturing  strategy  has  been  refined  and  integrated  with  the  risk  management 
plan.  Identification  of  enabling/critical  technologies  and  components  is  complete.  Prototype  materials,  tooling 
and  test  equipment,  as  well  as  personnel  skills  have  been  demonstrated  on  components  in  a  production 
relevant  environment,  but  many  manufacturing  processes  and  procedures  are  still  in  development. 
Manufacturing  technology  development  efforts  have  been  initiated  or  are  ongoing.  Producibility  assessments  of 
key  technologies  and  components  are  ongoing.  A  cost  model  has  been  constructed  to  assess  projected 
manufacturing  cost. 

Technology 

Development 

Phase 

6 

Capability  to 
produce  a 
prototype 

system  or 
subsystem  in  a 
production 

relevant 

environment 

This  MRL  is  associated  with  readiness  for  a  Milestone  B  decision  to  initiate  an  acquisition  program  by  entering 
into  the  Engineering  and  Manufacturing  Development  (EMD)  Phase.  It  is  normally  seen  as  the  level  of 
manufacturing  readiness  that  denotes  acceptance  of  a  preliminary  system  design.  An  initial  manufacturing 
approach  has  been  developed.  The  majority  of  manufacturing  processes  have  been  defined  and  characterized, 
but  there  are  still  significant  engineering  and/or  design  changes  in  the  system  itself.  However,  preliminary 
design  has  been  completed  and  producibility  assessments  and  trade  studies  of  key  technologies  and 
components  are  complete.  Prototype  manufacturing  processes  and  technologies,  materials,  tooling  and  test 
equipment,  as  well  as  personnel  skills  have  been  demonstrated  on  systems  and/or  subsystems  in  a  production 
relevant  environment.  Cost,  yield  and  rate  analyses  have  been  performed  to  assess  how  prototype  data 
compare  to  target  objectives,  and  the  program  has  in  place  appropriate  risk  reduction  to  achieve  cost 
requirements  or  establish  a  new  baseline.  This  analysis  should  include  design  trades.  Producibility 
considerations  have  shaped  system  development  plans.  The  Industrial  Capabilities  Assessment  (ICA)  for 

Milestone  B  has  been  completed.  Long-lead  and  key  supply  chain  elements  have  been  identified. 

Technology 

Development 

Phase 

leading  to  a 

MS-B 

decision 
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Manufacturing  Readiness  Level  Definitions,  Descriptions,  and  Phase 

MRL 

Definition 

Description 

Phase 

7 

Capability  to 
produce 
systems, 
subsystems, 

or 

components 

in  a 

production 

representativ 

e 

environment 

This  level  of  manufacturing  readiness  is  typical  for  the  mid-point  of  the  Engineering  and  Manufacturing 
Development  (EMD)  Phase  leading  to  the  Post-  CDR  Assessment.  Technologies  should  be  on  a  path  to  achieve 
TRL  7.  System  detailed  design  activity  is  nearing  completion.  Material  specifications  have  been  approved  and 
materials  are  available  to  meet  the  planned  pilot  line  build  schedule.  Manufacturing  processes  and 
procedures  have  been  demonstrated  in  a  production  representative  environment.  Detailed  producibility 
trade  studies  are  completed  and  producibility  enhancements  and  risk  assessments  are  underway.  The  cost 
model  has  been  updated  with  detailed  designs,  rolled  up  to  system  level,  and  tracked  against  allocated 
targets.  Unit  cost  reduction  efforts  have  been  prioritized  and  are  underway.  Yield  and  rate  analyses  have 
been  updated  with  production  representative  data.  The  supply  chain  and  supplier  quality  assurance  have 
been  assessed  and  long-lead  procurement  plans  are  in  place. 

Manufacturing  plans  and  quality  targets  have  been  developed.  Production  tooling  and  test  equipment  design 
and  development  have  been  initiated. 

Engineering 

and 

Manufacturing 
Development 
Phase  leading 

to  a  Pro  Critical 

Design  Review 

assessment 

8 

Pilot  line 

capability 

demonstrated 

;  Ready  to 
begin  Low 

Rate  Initial 

Production 

This  level  is  associated  with  readiness  for  a  Milestone  C  decision,  and  entry  into  Low  Rate  Initial  Production 
(LRIP).  Technologies  should  have  matured  to  at  least  TRL  7.  Detailed  system  design  is  complete  and 
sufficiently  stable  to  enter  low  rate  production.  All  materials,  manpower,  tooling,  test  equipment  and 
facilities  are  proven  on  pilot  line  and  are  available  to  meet  the  planned  low  rate  production  schedule. 
Manufacturing  and  quality  processes  and  procedures  have  been  proven  in  a  pilot  line  environment  and  are 
under  control  and  ready  for  low  rate  production.  Known  producibility  risks  pose  no  significant  challenges  for 
low  rate  production.  Cost  model  and  yield  and  rate  analyses  have  been  updated  with  pilot  line  results. 

Supplier  qualification  testing  and  first  article  inspection  have  been  completed.  The  Industrial  Capabilities 
Assessment  for  Milestone  C  has  been  completed  and  shows  that  the  supply  chain  is  established  to  support 

LRIP. 

Engineering 

and 

Manufacturing 
Development 
Phase  leading 

to  a  Milestone 

C  decision 
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Manufacturing  Readiness  Level  Definitions,  Descriptions,  and  Phase 

MRL 

Definition 

Description 

Phase 

9 

Low  rate 

production 
demonstrated; 
Capability  in  place 
to  begin  Full  Rate 

Production 

At  this  level,  the  system,  component  or  item  has  been  previously  produced,  is  in  production,  or 
has  successfully  achieved  low  rate  initial  production.  Technologies  should  have  matured  to  TRL 

9.  This  level  of  readiness  is  normally  associated  with  readiness  for  entry  into  Full  Rate 

Production  (FRP).  All  systems  engineering/design  requirements  should  have  been  met  such 
that  there  are  minimal  system  changes.  Major  system  design  features  are  stable  and  have  been 
proven  in  test  and  evaluation.  Materials,  parts,  manpower,  tooling,  test  equipment  and  facilities 
are  available  to  meet  planned  rate  production  schedules.  Manufacturing  process  capability  in  a 
low  rate  production  environment  is  at  an  appropriate  quality  level  to  meet  design  key 
characteristic  tolerances.  Production  risk  monitoring  is  ongoing.  LRIP  cost  targets  have  been 
met,  and  learning  curves  have  been  analyzed  with  actual  data.  The  cost  model  has  been 
developed  for  FRP  environment  and  reflects  the  impact  of  continuous  improvement. 

Production 

and 

Deployment 
Phase  leading 

a  Full  Rate 

Production 

(FRP)  decision 

10 

Full  Rate 

Production 

demonstrated  and 

lean  production 
practices  in  place 

This  is  the  highest  level  of  production  readiness.  Technologies  should  have  matured  to  TRL  9. 
This  level  of  manufacturing  is  normally  associated  with  the  Production  or  Sustainment  phases  of 
the  acquisition  life  cycle. 

Engineering/design  changes  are  few  and  generally  limited  to  quality  and  cost  improvements. 
System,  components  or  items  are  in  full  rate  production  and  meet  all  engineering,  performance, 
quality  and  reliability  requirements. 

Manufacturing  process  capability  is  at  the  appropriate  quality  level.  All  materials,  tooling, 
inspection  and  test  equipment,  facilities  and  manpower  are  in  place  and  have  met  full  rate 
production  requirements.  Rate  production  unit  costs  meet  goals,  and  funding  is  sufficient  for 
production  at  required  rates.  Lean  practices  are  well  established  and  continuous  process 
improvements  are  ongoing. 

Full  Rate 

Production  / 

Sustainment 

Wichipn  Chapter 
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System  Readiness  Level  Definitions,  Descriptions,  and  Level  of  Integration 

SRL 

Definition 

Description 

Level  of  Integration 

7 

System  prototype 

demonstration  in  an 

operational 

environment 

Prototype  near,  or  at,  planned  operational  system.  Represents  a  major 
step  up  from  SRL  6,  requiring  demonstration  of  an  actual  system 
prototype  in  an  operational  environment  such  as  an  aircraft,  vehicle,  or 
space,  including  interaction  with  external  systems. 

Fully  integrated  with  prototype 
System  interfaces  qualified  in  an 
operational  environment. 

8 

Actual  system 
completed  and 
qualified  through 

test  and 

demonstration 

System  has  been  proven  to  work  in  its  final  form  and  under  expected 
conditions,  including  integration  with  external  systems.  In  almost  all 
cases,  this  SRL  represents  the  end  of  true  system  development. 

Examples  include  test  and  evaluation  of  the  system  in  its  intended 
context  and  operational  architecture  to 
determine  if  it  meets  design  specifications. 

Final  production  design 
validated  demonstrating 

internal  and  external 

integration. 

9 

Actual  system 
proven  through 

successful  mission 

operations 

Actual  application  of  the  system  in  its  final  form  and  under  mission 
conditions,  such  as  those  encountered  in  operational  test  and  evaluation. 
Examples  include  operational  test  and  evaluation.  Examples  include 
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System  Readiness  Level  Definitions,  Descriptions,  and  Level  of  Integration 

SRL 

Definition 

Description 

Level  of  Integration 

1 

Basic  principles 

observed  and 

reported. 

Lowest  level  of  system  readiness.  Scientific  research  begins  to  be  translated  into 
applied  research  and  development  (R&D).  Examples  might  include  paper  studies 
of  a  system's  basic  properties. 

Interface  requirements  understood  at 
concept  level  only.  Impact  on  other 
systems  is  understood  at  a  concept 
level  only. 

2 

System  concept  and/or 
application  formulated. 

Invention  begins.  Once  basic  principles  are  observed,  practical  applications  can 
be  invented.  Applications  are  speculative,  and  there  may  be  no  proof  or 
detailed  analysis  to  support  the  assumptions.  Examples  are  limited  to  analytic 

studies. 

3 

Analytical  and 
experimental  critical 
function  and/or 
characteristic  proof  of 
concept. 

Active  R&D  is  initiated.  This  includes  analytical  studies  and  laboratory  studies  to 
physically  validate  the  analytical  predictions  of  separate  elements  of  the  system. 
Examples  might  include  COTS  components  that  are  not  yet  integrated  or 
representative. 

Analytical  assessment  conducted  to 
establish  interface  requirements. 

4 

Componentand/or 
breadboard  validation 

in  a  laboratory 

environment. 

Basic  system  components  are  integrated  to  establish  that  they  will  work 
together.  This  is  relatively  "low  fidelity"  compared  with  the  eventual  system. 
Examples  include  integration  of  "ad  hoc"  hardware  in  the  laboratory. 

Interface  requirements  specified  and 
understood.  The  likely  impact  on 
interfaced  systems  is  generally 

understood. 

5 

Componentand/or 
breadboard  validation 

in  relevant 

environment 

Fidelity  of  system  components  increases  significantly.  The  basic  system 
components  are  integrated  with  reasonably  realistic  supporting  elements  so  the 

total 

system  can  be  tested  in  a  simulated  environment.  Examples  include  "high- 
fidelity"  laboratory  integration  of  components  into  system  elements. 

Interfaces  partially  demonstrated  at 
System/Subsystem  level  in  a 
synthetic  environment.  Impact  on 
other  system  is  understood,  specified 
and  quantified. 

W  :hipnC 

III 

5 

D 

[mtmlnd 

System/subsystem 
model  or  prototype 
^gnonstration  in  a 

Representative  model  or  prototype  system,  which  is  demonstrated  in  a  well- 
simulated  operational  environment,  including  interaction  with  simulations  of  key 
external  systems. 

Interfaces  demonstrated  at  sy 
level  in  a  synthetic  /  high  fidel 
environment.  f^a 

stem 

ity 

relevant  environment- 
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